[Bugfix][plugin] fla crash on plugin#27322
Merged
mgoin merged 1 commit intovllm-project:mainfrom Nov 3, 2025
Merged
Conversation
ILikeIneine
changed the title
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request addresses a crash in Flash-Linear-Attention (FLA) operations when used with plugins. The changes in vllm/model_executor/layers/fla/ops/utils.py are well-reasoned and effective. By leveraging current_platform.is_cuda_alike(), the code now correctly identifies CUDA-compatible platforms (including plugins) and sets the device library appropriately. Adding a None default to getattr is a good defensive measure that prevents crashes on other platforms like CPU, making the utility more robust. The fix is correct and improves the overall stability of FLA operations in diverse environments.
NickLucche
reviewed
Oct 22, 2025
Collaborator
NickLucche
left a comment
NickLucche
left a comment
There was a problem hiding this comment.
I think this looks fine but I don't have the context on fla.
Perhaps @youkaichao can take a quick look at it.
ILikeIneine
force-pushed
the
fix-fla-crash-on-plugin
branch
from
873d0a9 to
b483cc9
Compare
mgoin
approved these changes
Nov 1, 2025
Member
mgoin
left a comment
mgoin
left a comment
There was a problem hiding this comment.
Looks simple enough to me. I believe the logic is kept the same for Nvidia and AMD, so nothing else changes for Intel or CPU
Signed-off-by: Hank <hcc.mayday@gmail.com>
ILikeIneine
force-pushed
the
fix-fla-crash-on-plugin
branch
from
59e4598 to
e379c89
Compare
ZhengHongming888
pushed a commit
to ZhengHongming888/vllm
that referenced
this pull request
Nov 8, 2025
ILikeIneine
added a commit
to MetaX-MACA/vLLM-metax
that referenced
this pull request
Nov 11, 2025
related: vllm-project/vllm/pull/27322 Signed-off-by: Hank <hcc.mayday@gmail.com>
ILikeIneine
added a commit
to MetaX-MACA/vLLM-metax
that referenced
this pull request
Nov 14, 2025
* support platform and remove kernel copy Signed-off-by: Hank <hcc.mayday@gmail.com> * update pre-commit Signed-off-by: Hank <hcc.mayday@gmail.com> * update version and requirements Signed-off-by: Hank <hcc.mayday@gmail.com> * update flashinfer Signed-off-by: Hank <hcc.mayday@gmail.com> * update build requirements Signed-off-by: Hank <hcc.mayday@gmail.com> * update attention backends Signed-off-by: Hank <hcc.mayday@gmail.com> * update patch Signed-off-by: Hank <hcc.mayday@gmail.com> * update quant_method Signed-off-by: Hank <hcc.mayday@gmail.com> * update fuse_moe (todo: fix mypy) Signed-off-by: Hank <hcc.mayday@gmail.com> * update `deepseek_v2.py`(todo: fix indexer kernel) Signed-off-by: Hank <hcc.mayday@gmail.com> * [feat] support bf16 cp_gather_indexer_k_cache kernel Signed-off-by: Xin Li <lixin1620@gmail.com> * [fix] fix type error in bf16_paged_mqa_logits Signed-off-by: leex404 <lixin1620@gmail.com> * [feat] add topk logits ops Signed-off-by: leex404 <lixin1620@gmail.com> * [fix] private memory size too large in `sample_recovered_tokens_kernel` (#115) * [fix] fix sample_recovered_tokens_kernel use too much private memory Signed-off-by: Xin Li <xin.li@metax-tech.com> * [fix] fix type error in bf16_paged_mqa_logits Signed-off-by: Xin Li <xin.li@metax-tech.com> * [chore] change file directory Signed-off-by: Xin Li <xin.li@metax-tech.com> --------- Signed-off-by: Xin Li <xin.li@metax-tech.com> Co-authored-by: Xin Li <xin.li@metax-tech.com> Signed-off-by: leex404 <lixin1620@gmail.com> * [fix] fix missing topk logits custom ops definition Signed-off-by: leex404 <lixin1620@gmail.com> * [fix] add custom gptq_shuffle ops Signed-off-by: leex404 <lixin1620@gmail.com> * [fix] fix compile error Signed-off-by: leex404 <lixin1620@gmail.com> * platform config update Signed-off-by: Hank <hcc.mayday@gmail.com> * update qwen2.5_vl model Signed-off-by: Hank <hcc.mayday@gmail.com> * [fix] fix torch not found maca device Signed-off-by: leex404 <lixin1620@gmail.com> * remove hotfixes patch for torch2.8 Signed-off-by: Hank <hcc.mayday@gmail.com> * remove needless patch related: vllm-project/vllm/pull/27322 Signed-off-by: Hank <hcc.mayday@gmail.com> * [feat] topk_softmax support renormalize and bf16 Signed-off-by: leex404 <lixin1620@gmail.com> * [fix] update fused_moe to fit v0.11.1 Signed-off-by: leex404 <lixin1620@gmail.com> * [fix] fix fused moe config log missing Signed-off-by: leex404 <lixin1620@gmail.com> * use flash_attn as vit attn backend on qwen_vl Signed-off-by: Hank <hcc.mayday@gmail.com> * update quant_conf registry Signed-off-by: Hank <hcc.mayday@gmail.com> * fix and apply latest pre-commit of v0.11.1 Signed-off-by: Hank <hcc.mayday@gmail.com> * [feat] Keep all AITER kernels in _aiter_ops Signed-off-by: leex404 <lixin1620@gmail.com> * fix pre-commit on type casting Signed-off-by: Hank <hcc.mayday@gmail.com> * [fix] fix DeepSeek import error Signed-off-by: leex404 <lixin1620@gmail.com> * [feat] update deepseek_v2 to fit v0.11.1 Signed-off-by: leex404 <lixin1620@gmail.com> --------- Signed-off-by: Hank <hcc.mayday@gmail.com> Signed-off-by: Xin Li <lixin1620@gmail.com> Signed-off-by: leex404 <lixin1620@gmail.com> Co-authored-by: Xin Li <xin.li@metax-tech.com> Co-authored-by: leex404 <lixin1620@gmail.com> Co-authored-by: leex404 <42941760+leex404@users.noreply.github.com>
devpatelio
pushed a commit
to SumanthRH/vllm
that referenced
this pull request
Nov 29, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
There's some problem while supporting fla on plugin.
While importing the
fla/ops/utils, it crashed on here.Since in plugin the
devicemight got their own value (here in vllm-metax ismaca) anddevice_torch_libstill need to be their own. (here in vllm-metax istorch.cuda)So I use is_cuda_alike and set default value to None on
getattrto make some compatibilities for handling the corner cases. The semantics should be consistent with the original code.Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.